Accessibility settings

This paper is in the following e-collection/theme issue:

Formative Evaluation of Digital Health Interventions (3450) Virtual Patients (84) Extended Reality, Virtual Reality and Virtual Worlds (804) Clinical Communication, Electronic Consultation and Telehealth (803) Adoption and Change Management of eHealth Systems (952) Decision Support for Health Professionals (1635)

Published on in Vol 10 (2026)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/71717, first published .
Additional Investigations and Referrals by Primary Care Physicians in Uncertain Clinical Situations: Cross-Sectional Study Using Virtual Patient–Based Scenarios

Additional Investigations and Referrals by Primary Care Physicians in Uncertain Clinical Situations: Cross-Sectional Study Using Virtual Patient–Based Scenarios

Additional Investigations and Referrals by Primary Care Physicians in Uncertain Clinical Situations: Cross-Sectional Study Using Virtual Patient–Based Scenarios

Authors of this article:

Mathieu Lorenzo1, 2 Author Orcid Image ;   Thierry Pelaccia2 Author Orcid Image ;   Emmanuel Triby3 Author Orcid Image ;   Yunus Can1 Author Orcid Image ;   Romain Krider1 Author Orcid Image ;   Hubert Maisonneuve4 Author Orcid Image ;   Nicolas Fernandez5 Author Orcid Image
Article Authors Cited by Tweetations Metrics

    1Department of General Medicine and Territorial Training, Faculty of Medicine, Midwifery, and Health Sciences, University of Strasbourg, 4 rue Kischleger, Strasbourg, France

    2Center for Training and Research in Health Sciences Education (CFRPS), Faculty of Medicine, Midwifery, and Health Sciences, University of Strasbourg, Strasbourg, Grand Est, France

    3Faculty of Education and Lifelong Learning (ESPE), University of Strasbourg, Strasbourg, France

    4University Institute for Primary Care, Faculty of Medicine, University of Geneva, Geneva, Switzerland

    5Center for Applied Pedagogy in Health Sciences (CPASS), University of Montreal, Montréal, QC, Canada

    Corresponding Author:

    Mathieu Lorenzo, MD, PhD


    Background: Uncertainty affects at least 20% of primary care consultations, possibly leading primary care physicians to order additional investigations or referrals, affecting cost-effectiveness and patient safety. Experience is a key determinant in making these orders, along with anxiety or physicians’ reactions to uncertainty. Past studies addressing the links between experience and additional investigations and referrals in uncertain situations have used questionnaires, database analyses, or interviews with general practitioners, but no study has used fully standardized conditions.

    Objective: This study aimed to examine the association between experience and orders for additional investigations and referrals in uncertain situations using a standardized virtual patient simulation.

    Methods: A cross-sectional study was conducted with 40 physicians stratified by sex and experience (<10 vs ≥10 years). Participants engaged in a simulated clinical scenario involving a man aged 69 years with atypical dyspnea designed to evoke diagnostic uncertainty. The virtual patient was presented via first-person video to assess the physicians’ decision-making process. Participants’ years of clinical experience, sex, age, place of practice, type of practice, number of in-office patients, duration of consultations, State-Trait Anxiety Inventory Form Y (STAI-Y) and Physician Reaction to Uncertainty scores, and diagnostic hypotheses were collected and analyzed using multivariate regression models.

    Results: The group with <10 years of experience had higher STAI-Y (mean 41.3, SD 6.8 vs mean 32.7, SD 8.2; P<.001) and Physician Reaction to Uncertainty (mean 20.7, SD 5.4 vs mean 14.4, SD 6.9; P=.002) scores. Participants with <10 years of professional experience ordered more additional investigations and referrals on average: 10.2 (SD 3.4; 95% CI 8.9-11.7) vs 8.1 (SD 3.7; 95% CI 6.5-9.9; P=.03). There was no effect on costs: €153.80 (US $173.45) vs €129.60 (US $146.16) (effect size 0.296, 95% CI −0.348 to 0.939; P=.23). Multivariate analysis showed an association between the number of additional investigations and referrals with age (relative risk 0.980, 95% CI 0.963‐0.997; adjusted P=.02) and mean STAI-Y score (relative risk 0.984, 95% CI 0.968‐0.999; adjusted P=.04) but not with experience (<10 vs ≥10 years; R2=0.308).

    Conclusions: Less experienced physicians do not appear to overly rely on additional investigations and referrals in uncertain situations, suggesting that clinical reasoning remains a well-preserved skill among the younger generation of physicians. Future research could explore interventions targeting physicians’ anxiety and tolerance to uncertainty as potential factors influencing requests for additional investigations and referrals. In practical terms, two avenues could be explored: specific training for supervisors to help them address uncertainty tolerance in the feedback they provide to trainees and the use of virtual consultations as a complement to traditional training, with particular attention to this aspect.

    JMIR Form Res 2026;10:e71717

    doi:10.2196/71717

    Keywords

    clinical decision-making; uncertainty; patient simulation; primary care anxiety; diagnostic tests; referral and consultation 


    In primary care, at least 20% of consultations are in situations of uncertainty [1,2]. In such situations, primary care physicians (PCPs) sometimes order additional investigations, such as biological testing or radiological analyses, or offer to refer patients to specialists to help decision-making [1,3-6]. Optimal management of additional investigations and referrals is an objective for cost-effectiveness and patient safety [7-10]. Numerous papers suggest that younger generations of practitioners may be losing clinical skills and using too many additional investigations and referrals [11-13].

    Experience appears to be a key determinant in additional investigations and referrals ordering [14-17]. In addition to experience, several factors have been singled out in the literature for their possible influence on additional investigations and referrals ordering in situations of uncertainty. Some relate to conditions of practice, such as the time available for each patient or the availability of tests or specialists [1,5,14,18,19], or the existence and provision of forms for requesting tests [20]. Others include societal factors, such as the fear of possible prosecution or the low cost of tests in some health care systems [5]; practitioners’ clinical reasoning [14,20-24]; practitioners’ emotional reaction to uncertainty, fear of regrets, and anxiety [5,14,19-21,23,25,26]; influence of the patient [1,5,14,26,27]; practitioners’ expectations of performance [5]; or practitioners’ sex [18]. Anxiety, in particular, had been shown to be correlated with additional investigations and referrals ordering in uncertainty [5,14,23].

    In studies addressing the links between experience and additional investigations and referrals in situations of uncertainty, data were obtained using questionnaires [23], database analyses [18], or interviews with general practitioners [5,14,15,26]. To our knowledge, no study has been conducted under fully standardized conditions. Standardization involves adhering to 4 key principles: all students complete the same task, all receive identical and clear instructions, no student benefits from an undue advantage, and a predetermined grading system is applied consistently across all participants [28]. This approach ensures a fair and equitable evaluation process while minimizing the influence of external factors, such as patient variability, on additional investigations and referrals performance. Thus, to gain a better understanding of the links between experience and additional investigations and referrals in decision-making in situations of uncertainty, we believe that using a standardized situation, presented by virtual patients, is a sound approach.

    The objective of this study was to explore the association between experience and additional investigations and referrals in uncertain situations, using virtual primary care consultations.


    Measurement Tool

    We have developed a computer tool (Hello-Clinical; University of Strasbourg) that makes it possible to simulate virtual consultations to assess the decision-making process of health care professionals for training or research purposes under standardized conditions. This study represented the first use of this tool.

    We divided our virtual clinical consultations into 5 stages [4] to collect the participants’ diagnostic hypotheses at each stage in an open text field without suggestions for responses. The virtual patients were presented through first-person video recordings. At the beginning, the patient is seen standing in the waiting room and greeting the physician with “good morning” after being called by the PCP. The first set of diagnostic hypotheses is collected at this stage based on initial cues such as the patient’s appearance and apparent health status.

    Next, the patient sits down at the PCP’s desk and explains the reason for the visit. A second set of diagnostic hypotheses is then collected.

    The PCP can then interact with the patient by typing questions in a text field. An algorithm automatically selects and displays the appropriate video response. For example, if the PCP asks, “Have you had a fever since you started feeling tired?” a video is shown of the patient responding, “No, I haven’t noticed any fever recently.”

    After another round of diagnostic hypothesis collection, the PCP may request parts of the clinical examination. For instance, if blood pressure is requested, the corresponding values appear on the screen. A final diagnostic hypothesis collection follows, after which the PCP can write the additional investigations and referrals deemed appropriate in an open text field.

    The Population Under Study

    We recruited a sample of community-based PCPs practicing ambulatory medicine in the Grand Est region of France, including urban and rural settings. Physicians worked in nonacademic practices, either solo or group based.

    The sampling strategy was predefined and stratified according to sex and professional experience. We aimed to include similar numbers of men and women to reflect the gender distribution of PCPs in the region, as well as comparable numbers of physicians with less than and more than 10 years of professional experience. The 10-year threshold was chosen based on previously published work on the development of expertise [29-33].

    Recruitment followed a structured snowball procedure [34]: 2 initial PCPs known by ML were contacted and agreed to participate. Each participant was then asked to systematically provide the contact details of 1 to 3 additional PCPs. Potential participants were contacted via phone or email using a standardized information procedure. Oral informed consent was obtained prior to data collection.

    Although the initial inclusion criterion was being an actively practicing PCP, recruitment was subsequently guided by ongoing monitoring of the stratification variables. Additional participants were specifically sought to meet predefined sex and experience quotas. Data collection took place between June 2017 and September 2017.

    Virtual Clinical Case

    We opted to use a clinical virtual vignette to simulate a scenario in which even experienced general practitioners would likely consider requesting additional investigations and referrals. Specifically, we designed the vignette around an atypical case of pulmonary embolism. The script portrayed a 69-year-old man presenting with progressive dyspnea over several days. The clinical presentation was intentionally vague, with the goal of creating diagnostic uncertainty. The goal was to contextualize the participants’ decision-making by attempting to reproduce an authentic context. Details of the main characteristics of the virtual patient are provided in Textbox 1.

    Participants were able to familiarize themselves with the software beforehand by tackling a tutorial script on a 60-year-old man with recent, unexplained asthenia.

    Textbox 1. Main characteristics of the virtual patient used in a cross-sectional study on experience and additional investigations and referrals requests by primary care physicians in uncertain situations using standardized virtual patient simulations in France in 2017.

    Medical history

    • Smoking ceased in 2012: 20 pack-years
    • High blood pressure since 2010
    • Right ulnar head fracture in 1989
    • Peritonitis in 1974

    Biometrics in the virtual case

    • 85 kg
    • 1.86 m
    • BMI 24.5 kg/m2
    • Blood pressure 132/76 mm Hg
    • Pulse 110 bpm

    Treatment

    • Hydrochlorothiazide 25 mg/day

    Variables Collected

    We identified various variables known to influence additional investigations and referrals ordering in situations of uncertainty [1,5,14,15,18,20,21,23]: participants’ sex, age, years of clinical experience, anxiety, place of practice, type of practice, average number of in-office patients, and average duration of consultations. Experience was defined as the number of years of practice since graduation. Place of practice (urban, rural, or semiurban), type of practice (alone or association), number of regular in-office patients, and average duration of consultation were related to practice conditions [5,14,15,18]. These data were collected through a questionnaire submitted after the simulated consultations. No confounding patient-related factors were collected, given the use of a standardized patient. Test availability was set to maximum in this virtual clinical case. We measured the participants’ anxiety levels using the State-Trait Anxiety Inventory Form Y (STAI-Y) scale [35,36] in its French version, observing only the state-anxiety dimension to save time. We selected this instrument because it is a widely used and validated measure of anxiety, with a validated French version available [35,37]. It has been used in PCP trainees [38]. We gathered responses to the French version of the Physician Reaction to Uncertainty (PRU) questionnaire [39,40]. This questionnaire has been used previously in PCP populations [40,41]. We then asked participants to rate the uncertainty and authenticity of the virtual clinical case from 1 (no uncertainty) to 5 (complete uncertainty) to verify if the case was an uncertain situation for them.

    ML, YC, and RK scored the severity of the final diagnostic hypotheses based on the French clinical classification of emergency patients (classification clinique des malades des urgences; CCMU):

    • CCMU 1: clinically stable, no immediate risk
    • CCMU 2: condition requiring prompt attention, but no immediate life-threatening risk
    • CCMU 3: potentially serious condition with a risk to life or function, requiring urgent care
    • CCMU 4: very serious condition with a short-term life-threatening prognosis
    • CCMU 5: deceased

    Additional investigations and referrals were collected after the final stage of diagnostic hypothesis generation, categorized, and assigned costs according to the French national health pricing system (Caisse Nationale de l’Assurance Maladie des Travailleurs Salariés; CNAMTS).

    Statistical Analysis

    In the absence of literature on the subject, we made an empirical hypothesis that PCPs with less than 10 years of experience would order a mean number of additional investigations and referrals of 10 (SD 6.0) and PCPs with 10 years or more of experience would order a mean number of additional investigations and referrals of 5 (SD 2.5). We then had to include 20 PCPs in each group to obtain a 90% power with a 5% type I error with an allocation ratio of 1:1. We used a 2-sample 2-tailed t test for the power analysis.

    To conduct the descriptive analysis of quantitative variables, for each variable, we compiled position parameters and dispersion parameters. To describe qualitative variables such as sex or type of practice, we compiled the headcounts and proportions for each modality in the sample.

    Comparisons of quantitative variables between groups were performed using a Poisson regression model for counting variables, a linear model for Gaussian variables, or a nonparametric Mann-Whitney-Wilcoxon test for non-Gaussian variables. For comparisons between more than 2 groups, we relied either on ANOVA (for the Gaussian data) or the Kruskal-Wallis test (non-Gaussian data). The correlation between the quantitative variables was measured using the Pearson correlation coefficient or the Spearman correlation coefficient.

    A multivariate analysis was performed with all variables. We used a stepwise selection method based on the minimization of the Akaike information criterion. Analyses were performed using R software (version 3.2.2; The R Project for Statistical Computing).

    Ethical Considerations

    We obtained ethics committee approval from the Ethical Reflection Group of the Groupe hospitalier Sélestat-Obernai on June 20, 2017, for this study (GH/SO 20062017CY/KR). Participants contacted via email received written study information, whereas those contacted via phone were informed orally. In all cases, oral informed consent was obtained prior to data collection. This approach aligns with French regulatory requirements, specifically a 2016 law (Décret n° 2016-1537 du 16 novembre 2016 relatif aux recherches impliquant la personne humaine [42]) that distinguishes between research involving human subjects and studies that do not fall under this category.

    Our study is classified as a noninterventional research not involving human subjects (as defined by Articles L. 1121‐1 and R. 1121‐1 of the French Public Health Code). For such studies, oral consent is considered sufficient, provided that participants are fully informed about the study’s purpose, procedures, and their rights—including the right to withdraw at any time. This approach is consistent with ethical guidelines for medical education research in France, where the emphasis is on ensuring clarity, transparency, and participant autonomy without imposing unnecessary administrative burdens.

    All data were anonymized. No compensation was provided to participants.


    Population

    In total, 40 PCPs participated in the study. Half of the participants were women. The mean age of participants was 43 (SD 13) years, and the median age was 37 years (range 31-56.8). The average number of years of experience was 15.1 (minimum 1 to maximum 40; SD 13.6). Half of the participants had more than 10 years of experience. The average STAI-Y score was 37 (SD 8.6; 95% CI 34.2‐39.8), indicating a low level of anxiety. The average PRU score was 17.6 (SD 6.9; 95% CI 15.4‐19.8). Participants rated the scenario’s uncertainty at 3.1 out of 5 (SD 0.9) and its authenticity at 4.4 out of 5 (SD 0.8).

    Participants had, on average, 3 final diagnostic hypotheses (SD 0.9; minimum 2 to maximum 5), with an average CCMU score of 2.5 (SD 0.4; minimum 1.5 to maximum 3.7). They requested an average of 10.3 additional investigations and referrals (SD 3.6; minimum 1 to maximum 16), with 34 different proposals. The average cost of additional investigations and referrals per participant was €141.80 (SD 81.6; min 17.5-max 440.5; at the time of the study, the conversion rate of US$ to € was 0.8867). Details are provided in Table 1.

    The group with less than 10 years of experience had, on average, fewer regular patients (158 vs 745; P<.001) and higher STAI-Y (mean 41.3, SD 6.8 vs mean 32.7, SD 8.2; P<.001) and PRU scores (mean 20.7, SD 5.4 vs mean 14.4, SD 6.9; P=.002).

    Both groups were equivalent in terms of gender, place and type of practice, uncertainty, authenticity of the case, mean CCMU scores, main diagnostic hypothesis’ CCMU score, and average duration of consultations (Table 1).

    Table 1. Univariate analysis of a group of primary care physicians with less than 10 years of experience vs more than 10 years in a cross-sectional study of experience and additional investigations and referral (AIR) requests in uncertain situations using standardized virtual patient simulations in France in 2017.
    Total (N=40)<10 years of experience (n=20)>10 years of experience (n=20)
    Age (years), mean (SD)43.1 (13.8)31.3 (2.5)54.9 (9.5)
    Women, n (%)20 (50)7 (35)13 (65)
    Experience (years), mean (SD)15.1 (13.6)3.6 (2.3)26.5 (9.9)
    Practice setting, n (%)
    Alone26 (65)14 (70)12 (60)
    Association14 (35)6 (30)8 (40)
    Practice location, n (%)
    Urban25 (62.5)11 (55)14 (70)
    Semiurban15 (37.5)9 (45)6 (30)
    Regular in-office patients, mean (SD)451 (413)158 (285.9)745 (443.9)
    Locum PCPsa, n (%)14 (35)12 (6)2 (10)
    Duration of consultations (minutes), n (%)
    <102 (5)1 (5)1 (5)
    10‐2032 (80)16 (80)16 (80)
    20‐306 (15)3 (15)3 (15)
    Uncertainty rating, mean (SD)3.1 (0.1)3.3 (0.8)2.9 (1.0)
    Authenticity rating, mean (SD)4.4 (0.8)4.5 (0.7)4.3 (0.9)
    STAI-Yb score, mean (SD)37.0 (8.6)41.3 (6.8)32.7 (8.2)
    PRUc score, mean (SD)17.6 (6.9)20.7 (5.4)14.4 (6.8)
    Number of AIRs, mean (SD)10.3 (3.7)11.5 (3.5)9.2 (3.7)
    Price of AIRs (€d), mean (SD)141.8 (81.6)153.8 (91.4)129.6 (70.8)
    Diagnostic hypotheses, mean (SD)3.0 (0.9)3.1 (1.0)2.9 (0.9)
    CCMUe score, mean (SD)2.5 (0.5)2.5 (0.5)2.5 (0.5)

    aPCP: primary care physicians

    bSTAI-Y: State-Trait Anxiety Inventory Form Y.

    cPRU: Physician Reaction to Uncertainty.

    dAt the time of the study, the conversion rate of US$ to € was 0.8867.

    eCCMU: classification clinique des malades des urgences.

    Primary End Point

    Participants with less than 10 years of professional experience ordered fewer additional investigations and referrals on average: 10.2 (SD 3.4; 95% CI 8.9‐11.7) vs 8.1 (SD 3.7; 95% CI 6.5‐9.9; P=.02). The difference was mostly in laboratory testing, such as C-reactive protein, liver and kidney function tests, and complete blood count. The effect size was 0.603 (95% CI −0.052 to 1.257). There was no effect on costs: €153.8 vs €129.6 (effect size 0.296, 95% CI −0.348 to 0.939; P=.23).

    In multivariate analysis, there was an association between the number of additional investigations and referrals with age (relative risk [RR] 0.980, 95% CI 0.963‐0.997; adjusted P=.02) and mean STAI-Y score (RR 0.984, 95% CI 0.968‐0.999; adjusted P=.04) but not with years of experience categorized in 10-year thresholds (R2=0.308; Table 2).

    Table 2. Final model of the number of additional investigations and referrals after stepwise backward selection in a cross-sectional study on experience and additional investigations and referrals requests by primary care physicians in uncertain situations using standardized virtual patient simulations in France in 2017.
    Adjusted relative risk (95% CI)Adjusted P value
    Experience (>10 years)1.042 (0.673‐1.611).85
    Age (years)0.980 (0.963‐0.997).02
    Number of regular in-office patients1.000 (0.999‐1.000).15
    STAI-Ya score0.984 (0.968‐0.999).04
    Sex (men)0.842 (0.667‐1.064).14
    Number of hypotheses1.093 (0.980‐1.218).11

    aSTAI-Y: State-Trait Anxiety Inventory Form Y.


    Principal Findings

    Having 10 or more years of experience was not associated with a lower number or costs of additional investigations and referrals ordered in our virtual patient scenario. The group with less than 10 years of experience had significantly higher STAI-Y and PRU scores than the group with more than 10 years of experience, and there was a statistically significant association between STAI-Y score, age, and the number of additional investigations and referrals ordered.

    Comparison With Previous Studies and Interpretation

    Unlike many past studies [5,8,14,15], we did not find that greater experience was associated with a decrease in the number of requests for additional investigations and referrals in primary care. There was an association between age and the number of requests in a multivariate analysis. However, neither the difference in the number of requests nor the association with age seemed clinically significant. The difference was an average of 2 additional requests, corresponding to a mean cost difference of less than €25, primarily related to common laboratory tests. Physicians with less than 10 years of experience generated only slightly more diagnostic hypotheses than those with more than 10 years of experience (3.1 vs 2.9). In light of these results, in this virtual study, younger PCPs seem to not significantly rely on additional investigations and referrals to reach a diagnosis in uncertain situations in primary care.

    Several factors may explain this discrepancy. First, the study sample may reflect a selection bias, as participation in simulation studies tends to attract clinicians who are more comfortable with digital environments and reflective practice [43]. Second, the use of standardized virtual patients removes key contextual pressures, such as time constraints, patient expectations, and perceived medico-legal risks, all of which are known to influence investigation and referral behaviors in real practice [44]. These contextual pressures might weigh more heavily on less experienced physicians. Third, differences in cognitive load may help explain the findings. According to cognitive load and dual process theories, novices allocate more resources to problem-solving processes, whereas experts rely more on pattern recognition [45-47]. In a virtual environment without external pressures, cognitive load may not lead younger physicians to request more additional investigations and referrals than their senior counterparts. Finally, psychological factors such as tolerance of uncertainty and anxiety—measured using the PRU and STAI-Y—are known to influence investigation behavior (Gerrity et al [48] and Simpkin and Schwartzstein [49]). Our data suggest that while younger PCPs reported higher anxiety and lower tolerance of uncertainty, these traits did not independently predict additional investigations and referrals use once contextual biases were controlled for, indicating that experience alone may be an oversimplified predictor.

    In contrast, age and anxiety seemed to be associated with additional investigations and referral requests. The small RR values (0.980 and 0.984) indicate a questionable clinical significance. There were significantly more locum PCPs in the group with less than 10 years of experience (n=12 vs n=2, respectively; P<.001). Locum PCPs frequently move from one practice to another. They may therefore lack this feedback loop, which is essential for enhancing diagnostic reasoning and understanding the typical clinical progression of common conditions [50]. This might be part of the explanation of why less experienced PCPs requested more additional investigations and referrals and had significantly higher PRU and STAI-Y scores. Hillen et al [51] conceptualized uncertainty tolerance (UT) as “the set of negative and positive psychological responses—cognitive, emotional, and behavioral—provoked by the conscious awareness of ignorance about particular aspects of the world”. UT could lead to decision deferral as part of a negative behavioral response to uncertainty [51]. This could help explain a higher use of additional investigations and referrals in a more anxious group. The perception of uncertainty could also lead to negative emotional responses, such as worry or fear [51]. Such responses may be part of defensive medical practice [44]. Research shows that physicians sometimes order additional investigations and referrals to reassure patients and themselves [5,20,21,26], but the effect is small on patients [52]. Although such orders may reassure physicians, it would be better to develop interventions aimed at reducing physicians’ anxiety directly rather than accept orders for additional investigations and referrals made simply for reassurance. From a medical education perspective, an effort should be made to improve the practices of primary care residents’ supervisors by offering specific feedback on experience aimed at building up positive responses of UT [14,49,51]. This may promote the optimal use of additional investigations and referrals in primary care by future clinicians.

    Many international guidelines insist on the necessity to rationally use additional investigations and referrals after a period of watchful waiting in the absence of red flags in situations such as fatigue or unexplained symptoms [53-56]. In the present case of dyspnea that had progressed for several days in a man aged 69 years, prompt additional investigations and referrals were justified. Participants’ requests were mostly adequate considering the 8 additional investigations and referrals to order in the pregraduated national learning objectives for acute dyspnea: complete blood count, blood electrolyte panel, blood glucose, brain natriuretic peptide or N-terminal prohormone of brain natriuretic peptide, D-dimers, electrocardiography, chest X-ray, and arterial blood gas analysis [57]. Our data suggest that progress could still be made on the rational use of additional investigations and referrals. Interventions such as the “five-step program for diagnostic test addiction” [58] could be used in a primary care setting: decide what diagnosis you are investigating, determine the pretest probability of the condition in question, decide whether to rule the condition in or out, decide what you will do if the test result is positive or negative, and ask whether ordering this test could hurt the patient. It would be interesting to explore the relationship between requests for additional investigations and referrals and clinical experience in scenarios where an experienced physician might choose not to rely on such requests. Such situations could provide valuable insights into how experience influences decision-making and the perceived necessity of additional tests, particularly in cases where clinical confidence may outweigh diagnostic uncertainty.

    From a clinical reasoning perspective, experience is conducive to the optimization of knowledge organization in the form of clinical scripts among PCPs (clinical scripts) [4,59]. We hypothesize that with more efficient clinical scripts, PCPs could better identify situations in which to order additional investigations and referrals, when to refrain from ordering them, and when to adopt a watchful, waiting approach [56]. In contrast, Watson et al [26] showed that PCPs developed their own mental shortcuts, including blood tests. Teaching clinical reasoning could incorporate practical exercises based on clinical cases that require decisions about whether to request additional investigations and referrals in situations of uncertainty [60].

    The way clinical reasoning is supervised may also positively influence students’ anxiety and tolerance of uncertainty. For instance, the SNAPPS (summarize, narrow, analyze, probe, plan, select) model has been shown to encourage learners to voice more diagnostic uncertainty during supervisory encounters [61-63]. This approach explicitly targets the inherent diagnostic uncertainty of clinical care and may therefore help improve the appropriateness of requests for additional investigations and referrals among future physicians.

    Limitations and Future Directions

    The authenticity of the case is limited by the extent of the software’s possibilities and current technical constraints. The software cannot capture certain types of information clinicians use in their reasoning, such as smells, and it is difficult for participants to fully detach from their usual practice context to engage with the simulated consultation environment [4]. The STAI-Y score only included the measurement of state-anxiety (not trait-anxiety). The impact of trait anxiety on additional investigations and referrals requests is not known. Moreover, the requests for additional investigations and referrals were self-reported and may not correctly predict real-life behavior. Experience was only defined by years of practice, although previous research has shown that the correlation between years of experience and clinical performance in medicine is often weak [32]. Qualitative insights on the participants’ expertise defined by peers rather than the number of years of experience could have enriched our comprehension of the topic. The snowball sampling method allowed us to respect the stratification of participant recruitment, ensuring that the sample aligned with the intended population characteristics. However, this method may have introduced some bias into our results, as it could increase the likelihood of finding associations specific to this population that may not exist in the general population [64]. Given that the target population was not particularly difficult to reach, future studies might consider using a larger sample with random sampling method to enhance generalizability and minimize potential biases. We chose to focus on only a portion of the potential determinants of additional investigations and referrals in our data collection for feasibility reasons.

    Conclusions

    Less experienced PCPs do not appear to overly rely on additional investigations and referrals in situations of uncertainty, suggesting that clinical reasoning remains a well-preserved skill among the younger generation of physicians. However, these findings should be interpreted with caution, as they may not fully account for the nuances in decision-making across varying levels of experience. Future research could explore interventions targeting physicians’ anxiety and tolerance of uncertainty as potential factors influencing additional investigations and referral requests. In practical terms, 2 avenues could be explored: specific training for supervisors to help them address UT in the feedback they provide to trainees and the use of virtual consultations as a complement to traditional training, with particular attention to this aspect. For instance, using methods such as SNAPPS could improve additional investigations and referral requests among future physicians by explicitly addressing clinical uncertainty. Additionally, it would be valuable to investigate scenarios where experienced physicians might consciously avoid additional investigations and referrals, providing further insights into the interplay between experience and test-ordering behavior in primary care.

    Acknowledgments

    We used generative artificial intelligence tools to assist with English language revision and to suggest stylistic improvements in selected sections of the manuscript, as well as in the responses to the reviewers and the editor. All content was carefully reviewed and validated by the authors. We would like to thank Professor Bernard Charlin for his assistance in revising the manuscript, and François Lefebvre from the Department of Public Health, Faculty of Medicine, Strasbourg, France, for his support in data analysis and manuscript revision.

    Funding

    The France-Quebec program and Initiative d'excellence (IDEX) program provided funding support throught the Samuel de Champlain program 2019. This article received support from the Maison Interuniversitaire des Sciences de l’Homme d’Alsace of the University of Strasbourg for translation.

    Data Availability

    The datasets generated and analyzed during this study are available on Google Drive [65].

    Authors' Contributions

    ML designed the study, collected and analyzed the data, and drafted the manuscript. RK helped design the study, collected and analyzed the data, and helped in drafting the manuscript. YC helped design the study, collected and analyzed the data, and helped in drafting the manuscript. FL helped design the study, analyzed the data, and critically revised the manuscript. ET, NF, TP, and HM helped design the study and critically revised the manuscript.

    Conflicts of Interest

    None declared.

    Checklist 1

    STROBE checklist.

    PDF File, 169 KB
    1. van der Weijden T, van Velsen M, Dinant GJ, van Hasselt CM, Grol R. Unexplained complaints in general practice: prevalence, patients' expectations, and professionals' test-ordering behavior. Med Decis Making. 2003;23(3):226-231. [CrossRef] [Medline]
    2. Kassirer JP. Our stubborn quest for diagnostic certainty. A cause of excessive testing. N Engl J Med. Jun 1, 1989;320(22):1489-1491. [CrossRef] [Medline]
    3. Howman M, Walters K, Rosenthal J, Ajjawi R, Buszewicz M. “You kind of want to fix it don’t you?” Exploring general practice trainees’ experiences of managing patients with medically unexplained symptoms. BMC Med Educ. Jan 25, 2016;16:27. [CrossRef] [Medline]
    4. Charlin B, Lubarsky S, Millette B, et al. Clinical reasoning processes: unravelling complexity through graphical representation. Med Educ. May 2012;46(5):454-463. [CrossRef] [Medline]
    5. van der Weijden T, van Bokhoven MA, Dinant GJ, van Hasselt CM, Grol RPTM. Understanding laboratory testing in diagnostic uncertainty: a qualitative study in general practice. Br J Gen Pract. Dec 2002;52(485):974-980. [Medline]
    6. Guthrie B. Why do general practitioners take blood? A cross-sectional study of use of blood tests in UK general practice. Eur J Gen Pract. 2001;7(4):138-160. [CrossRef]
    7. Sah S, Elias P, Ariely D. Investigation momentum: the relentless pursuit to resolve uncertainty. JAMA Intern Med. May 27, 2013;173(10):932-933. [CrossRef] [Medline]
    8. Logan RL, Scott PJ. Uncertainty in clinical practice: implications for quality and costs of health care. Lancet. Mar 2, 1996;347(9001):595-598. [CrossRef] [Medline]
    9. Blank L, Baxter S, Woods HB, et al. Referral interventions from primary to specialist care: a systematic review of international evidence. Br J Gen Pract. Dec 2014;64(629):e765-e774. [CrossRef] [Medline]
    10. Akbari A, Mayhew A, Al-Alawi MA, et al. Interventions to improve outpatient referrals from primary care to secondary care. Cochrane Database Syst Rev. Oct 8, 2008;2008(4):CD005471. [CrossRef] [Medline]
    11. Feddock CA. The lost art of clinical skills. Am J Med. Apr 2007;120(4):374-378. [CrossRef] [Medline]
    12. Bordage G. Where are the history and the physical? CMAJ. May 15, 1995;152(10):1595-1598. [Medline]
    13. Faustinella F, Jacobs RJ. The decline of clinical skills: a challenge for medical schools. Int J Med Educ. Jul 13, 2018;9:195-197. [CrossRef] [Medline]
    14. Lorenzo M. Tolérance à l’incertitude en médecine générale: une démarche de théorisation ancrée à partir de 14 entretiens. Université de Strasbourg; 2013. URL: https://www.sudoc.abes.fr/cbs/DB=2.1//SRCH?IKT=12&TRM=168128268 [Accessed 2026-02-21]
    15. Bugter-Maessen AM, Winkens RA, Grol RP, et al. Factors predicting differences among general practitioners in test ordering behaviour and in the response to feedback on test requests. Fam Pract. Jun 1996;13(3):254-258. [CrossRef] [Medline]
    16. Alam R, Cheraghi-Sohi S, Panagioti M, Esmail A, Campbell S, Panagopoulou E. Managing diagnostic uncertainty in primary care: a systematic critical review. BMC Fam Pract. Aug 7, 2017;18(1):79. [CrossRef] [Medline]
    17. Gardner NP, Gormley GJ, Kearney GP. Learning to navigate uncertainty in primary care: a scoping literature review. BJGP Open. Jul 29, 2024;8(2):BJGPO.2023.0191. [CrossRef] [Medline]
    18. Kristiansen IS, Hjortdahl P. The general practitioner and laboratory utilization: why does it vary? Fam Pract. Mar 1992;9(1):22-27. [CrossRef] [Medline]
    19. Forrest CB, Nutting PA, von Schrader S, Rohde C, Starfield B. Primary care physician specialty referral decision making: patient, physician, and health care system determinants. Med Decis Making. 2006;26(1):76-85. [CrossRef] [Medline]
    20. Zaat JO, van Eijk JT, Bonte HA. Laboratory test form design influences test ordering by general practitioners in the Netherlands. Med Care. Mar 1992;30(3):189-198. [CrossRef] [Medline]
    21. Koch H, van Bokhoven MA, ter Riet G, et al. What makes general practitioners order blood tests for patients with unexplained complaints? A cross-sectional study. Eur J Gen Pract. 2009;15(1):22-28. [CrossRef] [Medline]
    22. Amar É, Pereira C. Les prescriptions des médecins généralistes et leurs déterminants n°440. République Française Drees. Nov 1, 2005. URL: https:/​/drees.​solidarites-sante.gouv.fr/​publications/​etudes-et-resultats/​les-prescriptions-des-medecins-generalistes-et-leurs-determinants [Accessed 2026-02-21]
    23. Allison JJ, Kiefe CI, Cook EF, Gerrity MS, Orav EJ, Centor R. The association of physician attitudes about uncertainty and risk taking with resource use in a Medicare HMO. Med Decis Making. 1998;18(3):320-329. [CrossRef] [Medline]
    24. Miyakis S, Karamanof G, Liontos M, Mountokalakis TD. Factors contributing to inappropriate ordering of tests in an academic medical department and the effect of an educational feedback strategy. Postgrad Med J. Dec 2006;82(974):823-829. [CrossRef] [Medline]
    25. Tsalatsanis A, Hozo I, Vickers A, Djulbegovic B. A regret theory approach to decision curve analysis: a novel method for eliciting decision makers’ preferences and decision-making. BMC Med Inform Decis Mak. Sep 16, 2010;10:51. [CrossRef] [Medline]
    26. Watson J, de Salis I, Banks J, Salisbury C. What do tests do for doctors? A qualitative study of blood testing in UK primary care. Fam Pract. Nov 16, 2017;34(6):735-739. [CrossRef] [Medline]
    27. Little P, Dorward M, Warner G, Stephens K, Senior J, Moore M. Importance of patient pressure and perceived pressure and perceived medical need for investigations, referral, and prescribing in primary care: nested observational study. BMJ. Feb 21, 2004;328(7437):444. [CrossRef] [Medline]
    28. Jouquan J. L’évaluation des apprentissages des étudiants en formation médicale initiale. Pédagogie Méd. Feb 2002;3(1):38-52. [CrossRef]
    29. Croskerry P, Sinclair D. Emergency medicine: a practice prone to error? CJEM. Oct 2001;3(4):271-276. [CrossRef] [Medline]
    30. Anders Ericsson A, Smith J. Toward a General Theory of Expertise: Prospects and Limits. Cambridge University Press; 1991. ISBN: 9780521404709
    31. Patel VL, Kaufman DR. Medical expertise, cognitive psychology of. In: Smelser NJ, Baltes PB, editors. International Encyclopedia of the Social & Behavioral Sciences. Pergamon; 2001:9515-9518. [CrossRef] ISBN: 9780080430768
    32. Norman GR, Grierson LE, Sherbino J, Hamstra SJ, Schmidt HG, Mamede S. Expertise in medicine and surgery. In: Ericsson KA, Hoffman RR, Kozbelt A, Williams AM, editors. The Cambridge Handbook of Expertise and Expert Performance. Cambridge University Press; 2018:331-355. [CrossRef]
    33. Ericsson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med. Oct 2004;79(10 Suppl):S70-S81. [CrossRef] [Medline]
    34. Parker C, Scott S, Geddes A. Snowball sampling. In: Atkinson P, Delamont S, Cernat A, Sakshaug JW, Williams RA, editors. SAGE Research Methods Foundations. SAGE Publications; 2019. URL: https://doi.org/10.4135/9781526421036831710 [Accessed 2026-02-21]
    35. Spielberger CD. State-Trait Anxiety Inventory for Adults. American Psychological Association; 2012. URL: https://psycnet.apa.org/doiLanding?doi=10.1037%2Ft06496-000 [CrossRef]
    36. Spielberger CD, Gorsuch RL, Lushene RE. Manual for the State-Trait Anxiety Inventory. Consulting Psychologists Press; 1970.
    37. Gauthier J, Bouchard S. Adaptation Canadienne-Française de la forme révisée du State–Trait Anxiety Inventory de Spielberger. Can J Behav Sci. 1993;25(4):559-578. [CrossRef]
    38. Kerrien M, Pougnet R, Garlantézec R, et al. Prévalences des troubles anxieux, de l’humeur et de la souffrance au travail des internes de médecine. Presse Méd. Apr 2015;44(4):e84-e91. [CrossRef]
    39. Gerrity MS, White KP, DeVellis RF, Dittus RS. Physicians’ reactions to uncertainty: refining the constructs and scales. Motiv Emot. Sep 1995;19(3):175-191. [CrossRef]
    40. Le Roux V. Validation Culturelle et Psychométrique Du PRU, Physicians’ Reactions to Uncertainy Scales, Questionnaire Américain de Mesure Des Réactions Affectives Des Médecins Face à l’incertitude [thesis]. Université de Brest; 2015.
    41. Cooke GP, Doust JA, Steele MC. A survey of resilience, burnout, and tolerance of uncertainty in Australian general practice registrars. BMC Med Educ. Jan 7, 2013;13:2. [CrossRef] [Medline]
    42. Décret n° 2016-1537 du 16 novembre 2016 relatif aux recherches impliquant la personne humaine. Légifrance. URL: https://www.legifrance.gouv.fr/jorf/id/JORFTEXT000033394083 [Accessed 2026-03-23]
    43. Cook DA, Hatala R, Brydges R, et al. Technology-enhanced simulation for health professions education: a systematic review and meta-analysis. JAMA. Sep 7, 2011;306(9):978-988. [CrossRef] [Medline]
    44. Vandersteegen T, Marneffe W, Vandijck D. Defensive medicine: implications for clinical practice, patients and healthcare policy. Acta Clin Belg. Dec 2015;70(6):396-397. [CrossRef] [Medline]
    45. Baessler F, Hornung TP, Schuster TJ, et al. Game-based learning in undergraduate medical education: evaluation of an interdisciplinary escape room. BMC Med Educ. Nov 15, 2025;25(1):1606. [CrossRef] [Medline]
    46. Sweller J. Cognitive load during problem solving: effects on learning. Cogn Sci. Apr 1988;12(2):257-285. [CrossRef]
    47. Pelaccia T, Tardif J, Triby E, Charlin B. An analysis of clinical reasoning through a recent and comprehensive approach: the dual-process theory. Med Educ Online. Mar 14, 2011;16. [CrossRef] [Medline]
    48. Gerrity MS, DeVellis RF, Light DW. Uncertainty and professional work: perceptions of physicians in clinical practice. Am J Sociol. Jan 1992;97(4):1022-1051. [CrossRef]
    49. Simpkin AL, Schwartzstein RM. Tolerating uncertainty - the next medical revolution? N Engl J Med. Nov 3, 2016;375(18):1713-1715. [CrossRef] [Medline]
    50. Simpkin AL, Vyas JM, Armstrong KA. Diagnostic reasoning: an endangered competency in internal medicine training. Ann Intern Med. Oct 3, 2017;167(7):507-508. [CrossRef] [Medline]
    51. Hillen MA, Gutheil CM, Strout TD, Smets EM, Han PK. Tolerance of uncertainty: conceptual analysis, integrative model, and implications for healthcare. Soc Sci Med. May 2017;180:62-75. [CrossRef] [Medline]
    52. Rolfe A, Burton C. Reassurance after diagnostic testing with a low pretest probability of serious disease: systematic review and meta-analysis. JAMA Intern Med. Mar 25, 2013;173(6):407-416. [CrossRef] [Medline]
    53. Medically unexplained symptoms. Royal College of Psychiatrists. URL: https:/​/www.​rcpsych.ac.uk/​mental-health/​mental-illnesses-and-mental-health-problems/​medically-unexplained-symptoms [Accessed 2026-02-21]
    54. Koch H, van Bokhoven MA, ter Riet G, et al. Ordering blood tests for patients with unexplained fatigue in general practice: what does it yield? Results of the VAMPIRE trial. Br J Gen Pract. Apr 2009;59(561):e93-e100. [CrossRef] [Medline]
    55. White B, Renzi C, Barclay M, Lyratzopoulos G. Underlying cancer risk among patients with fatigue and other vague symptoms: a population-based cohort study in primary care. Br J Gen Pract. Jan 26, 2023;73(727):e75-e87. [CrossRef] [Medline]
    56. Wilson J, Morgan S, Magin PJ, van Driel ML. Fatigue--a rational approach to investigation. Aust Fam Physician. Jul 2014;43(7):457-461. [Medline]
    57. Collège des enseignants de pneumologie. Livret d’apprentissage (LiSA) EDN 2025 - item 203. URL: https://cep.splf.fr/wp-content/uploads/2023/07/ITEM_203_DYSPNEE_2023.pdf [Accessed 2026-3-23]
    58. Campbell SG, Innes GD, Magee KD, Elnenaei MO, Rowe BH. A five-step program for diagnostic test addiction. CJEM. Sep 2019;21(5):576-579. [CrossRef] [Medline]
    59. Charlin B, Boshuizen HPA, Custers EJ, Feltovich PJ. Scripts and clinical reasoning. Med Educ. Dec 2007;41(12):1178-1184. [CrossRef] [Medline]
    60. Chamberland M, Hivon R. Les compétences de l’enseignant clinicien et le modèle de rôle en formation clinique. Pedagog Medicale. May 2005;6(2):98-111. [CrossRef]
    61. Wolpaw T, Côté L, Papp KK, Bordage G. Student uncertainties drive teaching during case presentations: more so with SNAPPS. Acad Med. Sep 2012;87(9):1210-1217. [CrossRef] [Medline]
    62. Pierce C, Corral J, Aagaard E, Harnke B, Irby DM, Stickrath C. A BEME realist synthesis review of the effectiveness of teaching strategies used in the clinical setting on the development of clinical skills among health professionals: BEME Guide No. 61. Med Teach. Jun 2020;42(6):604-615. [CrossRef] [Medline]
    63. Sawanyawisuth K, Schwartz A, Wolpaw T, Bordage G. Expressing clinical reasoning and uncertainties during a Thai internal medicine ambulatory care rotation: does the SNAPPS technique generalize? Med Teach. Apr 2015;37(4):379-384. [CrossRef] [Medline]
    64. Patton MQ. Qualitative Research & Evaluation Methods: Integrating Theory and Practice. SAGE Publications; 2014. ISBN: 9781483301457
    65. Google Drive Repository. URL: https://docs.google.com/spreadsheets/d/1wCPAE7T39J-G04LhOtntA8JFDRtbZ0OehRfAlY6Wc6o/edit?usp=sharing

    ‎
    PCP: primary care physician
    PRU: Physician Reaction to Uncertainty
    RR: relative risk
    SNAPPS: summarize, narrow, analyze, probe, plan, select
    STAI-Y: State-Trait Anxiety Inventory Form Y
    UT: uncertainty tolerance

    Edited by Amaryllis Mavragani; submitted 24.Jan.2025; peer-reviewed by Joseph Walsh, Shankha Shubhra Goswami, Yoshiyuki Ohira; final revised version received 06.Feb.2026; accepted 10.Feb.2026; published 30.Mar.2026.

    Copyright

    © Mathieu Lorenzo, Thierry Pelaccia, Emmanuel Triby, Yunus Can, Romain Krider, Hubert Maisonneuve, Nicolas Fernandez. Originally published in JMIR Formative Research (https://formative.jmir.org), 30.Mar.2026.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.